Signal adaptive filtering method, signal adaptive filter and computer readable medium for storing program therefor

ABSTRACT

A signal adaptive filtering method for reducing blocking effect and ringing noise, a signal adaptive filter, and a computer readable medium. The signal adaptive filtering method capable of reducing blocking effect and ringing noise of image data when a frame is composed of blocks of a predetermined size includes the steps of: (a) generating blocking information for reducing the blocking effect and ringing information for reducing the ringing noise, from coefficients of predetermined pixels of the upper and left boundary regions of the data block when a frame obtained by deconstructing a bitstream image data for inverse quantization is an intraframe; and (b) adaptively filtering the image data passed through inverse quantization and inverse discrete cosine transform according to the generated blocking information and ringing information. Therefore, the blocking effect and ringing noise can be eliminated from the image restored from the block-based image, thereby enhancing the image restored from compression.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. application Ser. No. 14/164,875 filed onJan. 27, 2014, which is a continuation of application Ser. No.13/948,057 (now U.S. Pat. No. 8,638,864), filed Jul. 22, 2013, which isa continuation of application Ser. No. 11/649,275 (now U.S. Pat. No.8,494,048), filed Jan. 4, 2007, which is a continuation of applicationSer. No. 10/669,729 (now U.S. Pat. No. 7,492,823) filed Sep. 25, 2003,which is a divisional of application Ser. No. 09/462,805 (now U.S. Pat.No. 6,631,162), filed Jan. 14, 2000, which is a National StageApplication filed under §371 of PCT Application No. PCT/KR 98/00169,filed Jun. 18, 1998, which claims the benefit of Korean PatentApplication No. 10-1997-0033253, filed Jul. 16, 1997, the entiredisclosures of which are hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to data filtering, and more particularly,to a signal adaptive filtering method for reducing blocking effect andringing noise, a signal adaptive filter and a computer readable mediumfor storing a program therefor.

BACKGROUND ART

Generally, picture encoding standards such as MPEG of the InternationalStandardization Organization (ISO) and H.263 recommended by theInternational Telecommunication Union (ITU) adopt block-based motionestimation and discrete cosine transform (DCT) blocks. When an image ishighly compressed, the block-based coding may cause the well-knownblocking effect. A typical blocking effect is grid noise in ahomogeneous area in which adjacent pixels have relatively similar pixelvalues. Another blocking effect is staircase noise which has a staircaseshape and is generated along the edges of the image. Also, ringing noiseis a typical Gibb's phenomenon occurring by truncation when thecoefficients of the DCT are quantized so as to highly compress theimage.

Grid noise shows traces of the block-based process at the edges betweenblocks when the compressed data is displayed on a screen after beingrestored. Thus, one can identify the edges between blocks. Also,staircase noise has a staircase shape at the edges of the image, so thatone can notice a bumpy edge on the image. Also, one can noticeoverlapping of images with a predetermined interval due to the ringingnoise.

In order to reduce the blocking effect and the ringing noise occurringwhen block-based coding is performed, several methods have beensuggested. According to H.261 encoding, a simple 3×3 low-pass filter(LPF) is used as a loop filter to reduce the blocking effect [“VideoCodec for Audiovisual Services at Px62 kbit/s”, CCITT RecommendationH.261, Dec. 14, 1990]. Also, a simple edge loop filter has beensuggested so as to reduce the blocking effect and mosquito noise [G.Bjontegaard, “A Simple Edge Loop Filter to Reduce Blocking and MosquitoNoise”, ISO/IEC JTC1/Sc29/WG11 MPEG96/0617, January, 1996, and “A SimpleEdge Loop Filter to Reduce Blocking and Mosquito Noise”, ITU SC15 LBCExpert Group ITU-LBC-96032, January, 1996]. The edge loop filter makeslinear values of two pixels adjacent to the block boundary and replacesthe two pixel values by the linearized values. Such edge loop filter canreduce the blocking effect but not the ringing noise. In order to reducethe ringing noise, a non-linear filter using a binary index has beensuggested [Y. Itoh, “Detail Preserving Nonlinear Filter using BinaryIndex,” ISO/IEC JTC1/SC29/WG11 MPEG95/0357, November, 1995]. However,the non-linear filter cannot reduce the blocking effect.

DISCLOSURE OF THE INVENTION

To solve the above problems, it is an object of the present invention isto provide an signal adaptive filtering method for reducing blockingeffect and ringing noise in high compression encoding, a signal adaptivefilter, and a computer readable medium.

According to an aspect of the present invention, there is provided asignal adaptive filtering method capable of reducing blocking effect andringing noise of image data when a frame is composed of blocks of apredetermined size, the method comprising the steps of: (a) generatingblocking information for reducing the blocking effect and ringinginformation for reducing the ringing noise, from coefficients ofpredetermined pixels of the upper and left boundary regions of the datablock when a frame obtained by deconstructing a bitstream image data forinverse quantization is an intraframe; and (b) adaptively filtering theimage data passed through inverse quantization and inverse discretecosine transform according to the generated blocking information andringing information.

Preferably, the step (a) further comprises the step of setting theblocking information and ringing information of the previous framecorresponding to a motion vector as blocking information and ringinginformation of the current frame if the frame is an interframe, andsetting the ringing information to “1” which represents the image datarequiring filtering if a residual signal of the inverse-quantizedcurrent block exists, and the blocking information and ringinginformation are determined according to coefficients of a pixel Alocated at the upper left corner of the block, a pixel B located to theright of the pixel A and a pixel C located below the pixel A.

Preferably, the blocking information is constituted of horizontalblocking information and vertical blocking information, and thehorizontal blocking information is set to “1” which means the image datarequiring filtering when only the coefficient of the pixel A is notequal to “0” or any coefficient of the pixels of the left boundaryregion of the block is not equal to “0”, and the vertical blockinginformation is set to “1” which means the image data requiring filteringwhen only the coefficient of the pixel A is not equal to “0” or anycoefficient of the pixels of the upper boundary region of the block isnot equal to “0”, and the ringing information is set to “1” which meansthe image data requiring filtering when any coefficient of the pixelsother than the pixels A, B and C of the block is not equal to “0”.

Preferably, the blocking information is constituted of horizontalblocking information and vertical blocking information, and thehorizontal blocking information is set to “1” which means the image datarequiring filtering when all coefficients of the pixels A, B and C ofthe block are not equal to “0” or any coefficient of the pixels of theleft boundary region of the block is not equal to “0”, and the verticalblocking information is set to “1” which means the image data requiringfiltering when all coefficients of the pixels A, B and C are not equalto “0” or any coefficient of the pixels of the upper boundary region ofthe block is not equal to “0”, and the ringing information is set to “1”which means the image data requiring filtering when any coefficient ofthe pixels other than the pixels A, B and C of the block is not equal to“0”.

Preferably, in order to reduce the blocking effect, in the step (b), thehorizontal (or vertical) filtering is performed using a weighted filterhaving a predetermined weighted value when the horizontal (or vertical)blocking information of the block is equal to “1” and the ringinginformation is equal to “0”, and when the horizontal (or vertical)blocking information of the block is not equal to “1” or the ringinginformation is not equal to “0”. an absolute value of the differencebetween adjacent pixels and a Q value used as a dividend for quantizingthe block are compared, and then filtering is performed with apredetermined value according to the result of the comparison.

According to another aspect of the present invention, there is provideda signal adaptive filter capable of reducing blocking effect and ringingnoise of image data when a frame is composed of blocks of apredetermined size, comprising: a mode flag checking unit for checking aflag to determine whether or not a frame is an intraframe or aninterframe when a bitstream image data is deconstructed for inversequantization; an intra filtering information generator for generatingblocking information for reducing the blocking effect and ringinginformation for reducing the ringing noise, from coefficients ofpredetermined pixels of the upper and left boundary regions of the datablock when the frame is determined as an intraframe by the mode flagchecking unit; an inter filtering information generator for setting theblocking information and ringing information of the previous framecorresponding to a motion vector as blocking information and ringinginformation of the current frame if the frame is an interframe, andsetting the ringing information to “1” if a residual signal of theinverse-quantized current block exists; and an adaptive filter foradaptively filtering the image data passed through an inverse quantizerand an inverse discrete cosine transformer according to the blockinginformation and the ringing information generated by the intra filteringinformation generator and the inter filtering information generator.

The invention may be embodied in a general purpose digital computer thatis running a program from a computer usable medium, including but notlimited to storage media such as magnetic storage media (e.g., ROM's,floppy disks, hard disks, etc.), optically readable media (e.g.,CD-ROMs, DVDs, etc.) and carrier waves (e.g., transmissions over theInternet). Hence, the present invention may be embodied as a computerusable medium.

According to still another aspect of the present invention, there isprovided a computer readable medium having embodied thereon a computerprogram for a signal adaptive filtering capable of reducing blockingeffect and ringing noise of image data when a frame is composed ofblocks of a predetermined size, wherein the signal adaptive filteringcomprises the steps of: (a) generating blocking information for reducingthe blocking effect and ringing information for reducing the ringingnoise, from coefficients of predetermined pixels of the upper and leftboundary regions of the data block when a frame obtained bydeconstructing a bitstream image data for inverse quantization is anintraframe; (b) setting the blocking information and ringing informationof the previous frame corresponding to a motion vector as blockinginformation and ringing information of the current frame if the frame isan interframe, and setting the ringing information to “1” whichrepresents the image data requiring filtering if a residual signal ofthe inverse-quantized current block exists; and (c) adaptively filteringthe image data passed through the inverse quantization and inversediscrete cosine transform based on the block according to the generatedblocking information and ringing information.

Also, there is provided a computer readable medium having embodiedthereon a computer program for a method of filtering ringing noisecaused when decoding a block-based compressed image data, wherein theringing noise filtering method comprises the steps of: (a) performing agradient operation on the block subjected to inverse quantization andinverse discrete cosine transform using predetermined one dimensionalhorizontal and vertical gradient operators; (b) generating a binary edgemap representing whether or not each pixel is an edge pixel, using anabsolute value of the difference between the gradient-operated value ofone pixel and the value of the adjacent pixel, and a Q value used as adividend for quantizing the block; and (c) performing a filtering byapplying a predetermined filter window to the generated binary edge map.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a signal adaptive filter for reducingblocking effect and ringing noise according to the present invention;

FIG. 2 is a flowchart illustrating a signal adaptive filtering methodaccording to a preferred embodiment of the present invention;

FIG. 3 shows an inverse-quantized block having 8×8 pixels;

FIG. 4 is a flowchart illustrating the step of generating informationused to filter an intraframe;

FIG. 5 is a flowchart illustrating the step of generating informationused to filter an interframe;

FIG. 6 shows the location of pixels adjacent to the block boundary forillustrating the filtering for reducing blocking effect; and

FIG. 7 shows the location of pixels to be processed in the currentblock.

BEST MODE FOR CARRYING OUT INVENTION

In FIG. 1, a signal adaptive filter for reducing blocking effect andringing noise according to the present invention includes a mode flagchecking unit 120, an intra filtering information generator 130, aninter filtering information generator 140 and an adaptive filter unit150. When bitstream image data is deconstructed for inversequantization, the mode flag checking unit 120 checks whether or not theframe is an intra frame or an inter frame. The intra filteringinformation generator 130 generates blocking information for reducingblocking effect and ringing information from coefficients ofpredetermined pixels of the upper and left boundary regions of the datablock when the frame is determined as an intra frame by the mode flagchecking unit 120. When the mode flag checking unit 120 determines theframe as an inter frame, the inter filtering information generator 140generates blocking information and ringing information of the previousframe corresponding to a motion vector as blocking information andringing information of the current frame. If a residual signal of theinverse-quantized current block exists, the ringing information is setto “1”. The adaptive filtering unit 150 adaptively filters the imagedata of the block which has passed an inverse quantizer (Q⁻¹) 100 and aninverse discrete cosine transformer (DCT⁻¹) 110 according to theblocking information and ringing information generated by the intrafiltering information generator 130 and the inter filtering informationgenerator 140.

Meanwhile, a signal adaptive filtering method according to a preferredembodiment of the present invention will be described. FIG. 2 is aflowchart illustrating the signal adaptive filtering method according tothe present invention. Bitstream image data coded by an encoder isdecoded by a decoder to be reproduced. To end this, the bitstream datais deconstructed and then inverse-quantized by the inverse quantizer 100(step 200) Here the image data is constituted of a plurality of frames,and each frame is constituted of a plurality of blocks. FIG. 3 shows aninverse-quantized block having 8×8 pixels which form the frame.

Before filtering the frame data with the inverse discrete cosinetransform (IDCT), a flag is checked in order to make a determination asto whether or not the frame is an intraframe or an interframe (step210). If the frame is an intraframe (step 220), information used tofilter the intraframe is generated (step 230). If the frame is aninterframe, information used to filter the interframe is generated (step240). Then. the frame data which has passed through the IDCT 110 areadaptively filtered according to the generated filtering information,thereby eliminating blocking effect and ringing noise (step 250).

FIG. 4 is a flowchart illustrating in detail the step of generatinginformation used to filter the intraframe. As shown in FIG. 4, if theframe is determined as an intraframe by the mode flag checking unit 120,the coefficient of pixel A of FIG. 3 is checked (step 400). If only thecoefficient of the pixel A is not equal to “0”, horizontal blockinginformation (HBI) and vertical blocking information (VBI) are set to “1”(step 410). If any coefficient of pixels (8 pixels including the pixelsA and B) belonging to the upper boundary region 300 of the block shownin FIG. 3 is not equal to “0” (step 420), the VBI is set to “1” (step430). Otherwise, the VBI is set to “0” (seep 440). Also, if anycoefficient of pixels (8 pixels including the pixels A and C) belongingto the left boundary region 310 of the block shown in FIG. 3 is notequal to “0” (step 450), the HBI is set to “1” (step 460) Otherwise, theHBI is set to “0” (seep 470).

After the HBI and VBI are set, ringing information (RI) used to filterthe ringing noise is generated. That is, if any coefficient of pixelsother than the pixels A, B and C of the block shown in FIG. 3 is notequal to “0” (step 480), the RI is set to “1” (step 490). Otherwise, theRI is set to “0” (step 495). Here, the HBI and VBI are set to “1” whenonly the coefficient of the pixel A is not equal to “0” (step 400).However, by setting the HBI and VBI to “1” even if all coefficients ofthe pixels A, B and C are not equal to “0”, favorable effect to someextent can be obtained when a signal adaptive filtering is performedlater.

FIG. 5 is a flowchart illustrating the step of generating informationused to filter the interframe. If the frame is determined as aninterframe by the mode flag checking unit 120, the HBI, VBI and RI ofthe intraframe are transferred to HBI, VBI and RI of the interframeaccording to the motion vector (step 500). Also, if a residual signalexists after motion compensation (step 510), the RI is updated (step520).

When the blocking information and ringing information for filtering aregenerated as described above, filtering is adaptively performedaccording to the information. First, a filtering method for reducingblocking effect will be described. The filtering for reducing theblocking effect is classified as one of horizontal filtering andvertical filtering. Here, the horizontal filtering will be explained.FIG. 6 shows the location of pixels adjacent to the block boundary, forillustrating the filtering for reducing the blocking effect. Adetermination is made as to whether or not the HBI and RI of blocks Iand J of FIG. 6 are equal to “0”. If the HBI and RI of blocks I and J ofFIG. 6 are equal to “0”, a weighted filtering is performed on pixels A,B, C, D, E and F of FIG. 6 using a 7-tap (1, 1, 1, 2, 1, 1, 1) low-passfilter (LPF).

If the HBI or the RI of blocks I and J of FIG. 6 is not equal to “0”, afiltering is performed on the pixels B, C, D and E using the followingalgorithm.

d = D − C If (ABS(d) ≦ Q) {   D = D − (d/2); C = C + (d/2);   d = E − D;  if (ABS(d) ≦ Q) E = E − (d/4)   d = C − B;   if (ABS(d) ≦ Q) B = B +(d/4); } else {  If (ABS(d/2) ≦ 2Q)   If (d > 0) {    D = D − (Q −ABS(d/2));    C = C + (Q − ABS(d/2));   }   else {    D = D + (Q −ABS(d/2));    C = C + (Q − ABS(d/2));   }   d = E − D;   if (ABD(d) ≦ Q)E = E − (d/4);   d = C − B;   if (ABS(d) ≦ Q) B = B − (d/4);  } }

In the above algorithm, ABS represents an absolute value, and Qrepresents a dividend used when the blocks forming the frame arequantized.

In detail, the absolute value (ABS(d)) of the difference (d) between thepixels D and C is equal to or less than Q, the current pixel value ofthe pixel D is set by subtracting d/2 from the current pixel value, andthe current pixel value of the pixel C is set by adding d/2 to thecurrent pixel value. Also, the absolute value (ABS(d)) of the difference(d) between the pixels E and D is equal to or less than Q, the currentpixel value of the pixels is set by subtracting d/4 from the currentpixel value. Also, if the absolute value (ABS(d)) of the difference (d)between the pixels C and B is equal to or less than Q, the current pixelvalues of the pixel B is ,set by subtracting d/4 from the current pixelvalue. In the same manner as the above, the pixel values of the pixelsB, C, D and E are set according to the algorithm other than the above,which is obvious to those skilled in the art, thus the explanationthereof will be omitted. Also, vertical filtering is performed accordingto the same principle as the horizontal filtering.

Next, a filtering method for reducing ringing noise will be described.First, the generated ringing information is checked. If the ringinginformation is set to “1”, the filtering is performed. Otherwise, thefiltering is not performed. To end this, edge pixels of the block whichhave been subjected to inverse quantization and IDCT are determined. Inorder to determine the edge pixels, a gradient operation is performed onthe blocks which have been subjected to inverse quantization and IDCTusing one-dimensional horizontal and vertical gradient operators.

Then, an absolute value of the difference between one gradient-operatedpixel value and the adjacent pixel value, and Q value used as a dividendwhen quantizing the block, are used to generate a binary edge maprepresenting the edge of each pixel. Here, the block has 8×8 pixels, andthe size of the binary edge map is represented as a two-dimensionalarray edge[10][10] as shown in FIG. 7.

In order to generate the binary edge map, vertical edge detection andhorizontal edge detection are performed. Algorithms for vertical edgeand horizontal edge detections are as follows.

-   -   /*Vertical edge detection*/    -   A1=ABS(PtrImage[0]−PtrImage[1]);    -   A2=ABS(PtrImage[0]−PtrImage[1]);    -   if (((A1>Th)&&(A2>Th))∥(A1>5*Th/2)∥(A2)>5*Th/2))        -   Edge[m][n]=1;/*edge*/    -   else {/* Horizontal edge detection*/        -   A′1=ABS(PtrImage[0]−PtrImage[width]);        -   A′2=ABS(PtrImage[0]−PtrImage[−width]);        -   if (((A′1>Th)&&(A′2>Th)∥(A′1>5*Th/2)∥(A′2)>5*Th/2))            -   Edge [m][n]=1;/*edge*/

For the vertical edge detection, an absolute value (A1) of thedifference between the gradient-operated results of the pixel(PtrImage[0]) on which a determination as to whether or not the pixel isan edge of the block is made, and the right pixel (PtrImage([1]) of thepixel (PtrImage[0]) is calculated. Then, an absolute value (A2) of thedifference between the gradient-operated results of the PtrImage[0] andthe left pixel (PtrImage[−1]) of the pixel (PtrImage[0]) is calculated.Then, a determination as to whether or not the pixel is an edge is madeaccording to the logic values obtained after the absolute values A1 andA2 are compared with a predetermined threshold value Th, and then theabove process is performed on all pixels of the block. The vertical edgedetection is performed according to a logical formula of(A1>Th)&&(A2>Th)∥(A1>5*Th/2)∥(A2)>5*Th/2). If the logical formula istrue, the pixel is determined as a vertical edge. Otherwise, the pixelis determined to not be a vertical edge.

The horizontal edge detection is performed according to the sameprinciple of the horizontal edge detection. First, an absolute value(A′1) of the difference between the gradient-operated results of thepixel (PtrImage[0]) on which a determination as to whether or not thepixel is edge of the block is made, and the lower pixel(PtrImage[width]) of the pixel (PtrImage[0]) is calculated. Then, anabsolute value (A′2) of the difference between the gradient-operatedresults of the pixel (PtrImage[0]) and the upper pixel(PtrImage[−width]) of the pixel (PtrImage[0]) is calculated. Then, adetermination as to whether or not the pixel is an edge is madeaccording to the logic values obtained after the absolute values A′1 andA′2 are compared with a predetermined threshold value Th, and then theabove process is performed on all pixels of the block. The horizontaledge detection is performed according to a logical formula of(A′1>Th)&&(A′2>Th)∥(A′1>5*Th/2)∥(A′2)>5*Th/2). If the logical formula istrue, the pixel is determined as a horizontal edge. Otherwise, the pixelis determined to not be a horizontal edge. Here, ‘&&’ represents logicalAND, and ‘∥’ represents logical OR.

Next, filtering is performed by applying a predetermined filter windowto the generated binary edge map. The filtering may be performed by ageneral filtering method by applying a filter window having apredetermined size. However, in this embodiment, filtering is notperformed if the central pixel of the filter window is an edge, whilethe filtering is performed if the central pixel is not an edge. Thefilter window may be a general filter window. In this embodiment, a4-connectivity filter window having five pixels arranged in a crossshape centering one central pixel, as shown in FIG. 7, is used. In FIG.7, X represents an edge pixel, and the regions other than the regionswith “X” represent non-edge pixels.

Also, if the filter window has no edge pixel, ordinary filtering isperformed, while weighted filtering is performed if the edge pixelexists. An example of weighted filtering is shown in FIG. 7. In FIG. 7,“<<” represents a shift to the left, and “>>” represents a shift to theright.

The invention may be embodied in a general purpose digital computer thatis running a program from a computer usable medium, including but notlimited to storage media such as magnetic storage media (e.g., ROM's,floppy disks, hard disks, etc.), optically readable media (e.g.,CD-ROMs, DVDs, etc.) and carrier waves (e.g., transmissions over theInternet). Hence, the present invention may be embodied as a computerusable medium having a computer readable program code unit embodiedtherein for signal adaptive filtering, the computer readable programcode means in the computer usable medium comprising: computer readableprogram code means for causing a computer to effect generating blockinginformation for reducing the blocking effect and ringing information forreducing the ringing noise, from coefficients of predetermined pixels ofthe upper and left boundary regions of the data block when a frameobtained by deconstructing a bitstream image data for inversequantization is an intraframe; computer readable program code means forcausing a computer to effect setting the blocking information andringing information of the previous frame corresponding to a motionvector as blocking information and ringing information of the currentframe if the frame is an interframe, and setting the ringing informationto “1” which represents the image data requiring filtering if a residualsignal of the inverse-quantized current block exists; and computerreadable program code means for causing a computer to effect adaptivelyfiltering the image data passed through the inverse quantization andinverse discrete cosine transform based on the block according to thegenerated blocking information and ringing information, for instance. Afunctional program, code and code segments, used to implement thepresent invention can be derived by a skilled computer programmer fromthe description of the invention contained herein.

INDUSTRIAL APPLICABILITY

As described above, the present invention can remove the blocking noiseand the ringing noise from an image restored from an image compressedbased on blocks, thereby enhancing the image restored from compression.

What is claimed is:
 1. An image data filtering method for reducing ablocking effect and noise, the method comprising: obtaining a data blockincluding transformed coefficients; checking whether all coefficients ofall pixels in the data block are equal to zero or not; generatingfiltering information indicating whether to perform deblocking filteringfor the data block according to whether all of the coefficients of allof the pixels in the data block are equal to zero or not; and filteringthe data block passed through inverse quantization and inverse transformaccording to the generated filtering information.